Modern vehicles (e.g., airplanes, boats, trains, cars, trucks, etc.) can include a vehicle event recorder in order to better understand the timeline of an anomalous event (e.g., an accident). A vehicle event recorder typically includes a set of sensors, e.g., video recorders, audio recorders, accelerometers, gyroscopes, vehicle state sensors, GPS (global positioning system), etc., that report data, which is used to determine the occurrence of an anomalous event. If an anomalous event is detected, then sensor data related to the event is recorded and transmitted to a vehicle data server for later review by a human reviewer. A driver score can be determined using the reviewed events (e.g., events determined to be caused by poor driving reduce the score for the driver). However, for the case of scoring driver performance to a simulated optimal driver model, data visibility is required at a granularity beyond the exception based video review.